Systems and methods for performing quality assurance on interactive television and software application data delivered via a network

ABSTRACT

A system and method for providing quality assurance for interactive television and software application data packages delivered via a network. By employing “code checks” that determine “code points” based on the original package, content or data packages may be checked for errors at any later point within the network path from the provider to the subscriber. The data package can be checked for errors by performing some or all of the code checks and comparing the current code point values to the original code point values. If the current values differ from the original values, then the data package has changed and an error is likely to have occurred. Further action may then be taken to determine the nature of the error and to respond to it.

This application claims priority to, and incorporates herein byreference in its entirety, U.S. Provisional Application No. 60/396,119,entitled “SYSTEMS AND METHODS FOR PERFORMING QUALITY ASSURANCE ONINTERACTIVE TELEVISION AND SOFTWARE APPLICATION DATA DELIVERED VIA ANETWORK,” which was filed on Jul. 15, 2002.

FIELD OF THE INVENTION

The present invention is directed to systems and methods for performingquality assurance on interactive television and software applicationdata that is delivered via a network. BACKGROUND OF THE INVENTION

Any data which is delivered over a network is vulnerable to havingerrors introduced into the data. This problem is particularly prevalentin networks with multiple components and especially where no singleentity controls all of the network components between the sender andreceiver of the data.

For example, networks which deliver interactive television have manycomponents which are likely to be geographically remote from one anotherand are frequently under the control of a multiple entities. An exampleof this kind of network is one where a content provider (e.g., Home BoxOffice® (HBO®)) has provided content (e.g., a movie) to an interactivetelevision service provider for delivery to subscribers using aVideo-On-Demand (VOD) system where the subscriber selects a movie usinga Graphical User Interface (GUI).

FIG. 1 shows an example of a network implementing a VOD service. Thecontent provider prepares and delivers a Movies-On-Demand (MOD) datapackage 100 to an interactive television service provider. The MODpackage 100 contains the audio and video for the movie as well as otherdata that is used to provide at least part of the GUI display that thesubscriber views when ordering a movie.

The interactive television service provider loads the MOD package intoan Asset Management/Business Management system such as the MediaPath™system by N2Broadband, Inc of Duluth, GA. The Asset Management Server(AMS) 110 maintains a catalogue of the content (e.g., movies in thiscase) that is available on the VOD service. The Business ManagementServer (BMS) 120 handles the business and billing aspects of the VODservice by, for example, applying business rules to perform subscriptionchecking and ensuring that subscribers are properly billed for their useof the service. The audio and video that are to be delivered to thesubscriber when a movie is ordered are then stored in one or more VODservers 140 (e.g., a MediaClusterm by SeaChange International ofMaynard, Mass.) that may be located remotely from the AMS 110 and BMS120 so as to serve geographically diverse subscribers. The interactionover the network between the subscribers and the VOD service is managedby a Digital Storage Management-Command and Control (DSM-CC) 130 datacarousel such as the Digital Network Control System (DNCS) byScientific-Atlanta, Inc. of Lawrenceville, Ga.

Each subscriber has a Set Top Box (STB) 160 which connects to thesubscriber's television 170 and connects to the DSM-CC 130, VOD server140, AMS 110, and BMS 120 via the “cable plant” 150 of the subscriber'scable provider (which may or may not be the same as the interactivetelevision service provider). This cable plant consists primarily of thevarious hardware and software components (e.g., network switches, etc)required to provide connectivity between the STB 160 and the VOD servicecomponents. The STB 160 may be, for example, a Scientific-AtlantaExplorer 3000® running PowerTV® operating system software and thePrasara user interface application.

When a subscriber accesses the VOD service to order a movie, thesubscriber is first presented with the GUI, including information fromthe MOD package 100 as provided by the AMS 110. The subscriber may thenrequest to order the movie. The request to order the movie is routedthrough the DSM-CC 130 to the BMS 120 to determine if the subscriber isauthorized to order the movie and ensuring that billing for the serviceis properly handled. Once the BMS 120 authorizes the movie orderrequest, a VOD server 140 streams the movie audio and video to the STB160 which decodes the data and plays the movie on the subscriber's TV170.

There are numerous opportunities at a variety of places in the networkfor errors to be introduced into the GUI that is displayed to the user,and even the movie data itself. For example, the interactive televisionservice provider may manually load the MOD package 100 into the AMS 120,possibly editing portions of the GUI data in the process, and errors maybe introduced into the data. In another example, errors may beintroduced simply through corrupted transmission from one component inthe network to another especially when data elements from the same datapackage are stored on different components in the system. Errors mayalso be introduced due to inconsistencies between the software indifferent models of STBs.

These errors may be as trivial as a misspelling in the title of themovie or the artwork to be displayed on the GUI may be corrupted. Theseerrors may also be much more serious such as the complete corruption ofthe movie's audio or video or the mismatching of different data elementssuch as misclassification of adults-only material as children's content.

Present quality assurance techniques rely on direct inspection by ahuman, which is not practical, or on dedicated testing systems such asthe TestQuest Pro™ system by TestQuest, Inc. of Minneapolis, Minn.,which are computationally intensive and cannot simply and easily addresserrors that are introduced during transmission to the subscriber.

SUMMARY

The present invention provides a system and method for providing qualityassurance for interactive television and software application datadelivered via a network. Example embodiments of the present inventionemploy “code checks” that determine “code points” (e.g., a value, code,message, marker or pattern used for integrity checking) based on theoriginal, error-free content as a way to check the content or datapackage for changes or errors at any later point within the network pathfrom the provider to the subscriber.

Using the code checks, at any point in the network path from theprovider to the subscriber, the data can be checked for changes orerrors by performing some or all of the code checks and comparing thecurrently determined code point values to the originally determined codepoint values. If the current values differ from the original or“desired” values, then the data has changed and an error is likely tohave occurred. Further action may then be taken to determine the natureof the change or error and to respond to it.

Code checks may ensure the integrity of a related set of data, thus forthe first time allowing a data provider to be confident that the relatedset of data is received in its intended form.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates one possible network in which the system and methodof the present invention could be implemented.

FIG. 2 illustrates one possible embodiment for establishing the desiredcode points of the system and method of the present invention.

FIG. 3 illustrates one possible embodiment for checking the code pointsof the system and method of the present invention.

FIG. 4 illustrates another possible embodiment of the system and methodof the present invention.

DETAILED DESCRIPTION

As shown in FIG. 2, in one possible embodiment of the present invention,when the MOD package 280 is being assembled at the content provider,code points 290 are determined for the “desired” (e.g., error-free)content. This content may include, for example, audio 240 and video 220that may be encoded using, for example, Dolby Labs' AC-3® encoding 250and MPEG (Motion Picture Experts Group) encoding 230. The content mayalso include still art 260, such as promotional artwork, encoded using,for example, JPEG (Joint Photographic Experts Group) encoding 270,textual data such as title, cast, and ratings information (e.g., G, PG,R, etc.), and metadata 200 that describes how the GUI is displayed onthe screen in, for example, XML format 210. A code point for each of theelements (e.g., audio, video, still art, textual data, ratingsinformation, metadata, etc) of the MOD package may be determined by acode check.

These code checks may be any of a variety of well known techniques forintegrity checking. One possible method is to calculate a hash code orchecksum for each of the data elements. Another possible technique is touse watermarks such those of Verance Corporation or Digimarc Corporationto embed a code point within an audio, video or graphic element. Simpletechniques may also be used as code checks, such as performing a spellcheck of the text within the content or determining a summary of thetext within the content to provide a code point. The set of desired codepoints which is determined from the code checks also may be protectedfrom tampering by, for example, digitally signing them.

The desired code points may be included within the content package as aseparate data field, embedded directly within the content or packagedseparately. The desired code points also may be calculated before, afteror during the process of assembling the content package.

As shown in FIG. 3, the MOD data package 300 may then be sent to theinteractive television provider, and the set of desired code points 390associated with the MOD data package 300 may be sent separately to acode point monitoring system 380 that may be implemented within theinteractive television network (e.g., as software running on one thenetwork elements described previously). Alternatively, the code pointmonitor may be implemented on a separate piece of hardware or network ofhardware such as a computer server or the like. The code point monitormay then access the data package at any number of places within thenetwork to verify the integrity of the data. For example, the code pointmonitor may check the data after the data package has been loaded intothe AMS 310 and BMS 320, before it is distributed to the VOD servers340. Alternatively, the code point monitor may check the data when it ispassing through components in the cable plant 350 on its way to the useror when it is passing through the DSM-CC 330. The code point monitor mayalso check the data when it has been received by the STB 360 beforebeing sent to the subscriber's TV 370.

When the code point monitor 380 is, for example, checking the data afterit has been loaded into the AMS 310, the code point monitor may requesta copy of the data package 300 from the AMS 310. The data package isthen matched to a particular set of desired code points 390 using, forexample, a unique identifier. Once the code point monitor 380 receivesthe data package 300, new code points are determined for the datapackage using the same code checks that were used at the contentprovider. The new code points are then compared to the set of desiredcode points 390 that the code point monitor 380 has received from thecontent provider either via the interactive television network or viasome other network. If the new code points are identical to the desiredcode points, then the data has not been altered since the desired codepoints were determined. If the new code points differ from the desiredcode points then the data has been altered and errors may have beenintroduced. By considering each code point individually, the code pointmonitor may then determine which data elements have been changeddepending on which code points are different from their desired values.For example, if the watermark that was in the audio signal has beenaltered or is not present then the audio has been altered in some way.In some cases, although the desired code points and new code points arenot identical, they may be sufficiently similar that the data may bedetermined to still be accurate enough for use. For example, if an audiosignal is altered, the watermark which was embedded in it will also bealtered. However, the code point monitor may determine that thealteration to the watermark is minor and thus the changes to the audiosignal are minor enough that the signal may still be used. By comparingthe entire set of code points, the code point monitor 380 may alsodetect if an entire data element is missing or if a data element hasbeen replaced with a data element that belongs in a different datapackage.

Errors may also be detected in the set of code points 390 itself. Forexample, if code points that have been digitally signed are tamperedwith, then an error may be determined to have occurred. An error mayalso be determined to have occurred if the code point monitor 380 hasnot received the set of desired code points 390 for the data package 300(e.g., the data package was supposed to include the desired code pointinformation and does not, or the code point monitor never received thedesired code point information through its normal means of doing so).

After an error or alteration has been detected, the code point monitormay take a variety of actions. For example, if the error is amisspelling in the text of the data package, the code point monitor 380may attempt to correct the error. In another example, the code pointmonitor 380 may request that the entire data package 300 or a particulardata element of the data package 300 be retransmitted to the particularcomponent that the code point monitor 380 is checking (e.g., the codepoint monitor 380 may request the content provider to send a new copy ofthe data package 300 or data element to the AMS 310 so that the error iscorrected before the data is passed on to the VOD servers 340). The codepoint monitor 380 may also prevent the content from be accessed (e.g.,when the classification has been altered) or simply prevent a portion ofthe content from being displayed (e.g., if the GUI graphics have beencorrupted, the text could still be displayed and the content accessed,but the graphic would not display). The code point monitor 380 may alsolog that the error has occurred or report the error to some othercomponent in the network.

For example, the code point monitor 380 may check the data package 300as it passes through the cable plant 350 on its way to the subscriber'sSTB 360. The individual data elements of the data package 300 have beensent from different components in the interactive television provider'snetwork (i.e., the metadata and other GUI information has been sent fromthe AMS 310 via the DSM-CC 330 while the audio and video have been sentfrom the VOD server 340). However, an error may have occurred becausewhile the metadata and GUI information are for an animated children'sprogram, the audio and video are for the television show “The Sopranos.”The code point monitor 380 performs the appropriate code check on eachof the data elements and then compares the determined current code pointvalues to the set of desired code point values 390. In performing thatcomparison, the code point monitor 380 discovers the mismatched contentbecause the current code points for the video and audio data elements donot match the audio and video code points in the set of desired codepoints 390 while the current metadata and GUI information code points domatch their respective desired code points. The code point monitor thenblocks the content from being viewed, thus preventing the possibilitythat a child would be exposed to inappropriate material.

An alternative embodiment of the system and method of the presentinvention is shown in FIG. 4. As shown in FIG. 4, a content package 440containing video, audio, metadata and artwork is delivered to a serverarchive 450. Control software 460 then calculates desired code pointvalues for the various data elements in the package and separates themfrom the content package 440. The content package 440 is then “playedout” to the users of the service via the cable television system. Whenthe content is received by the user's STB, the metadata 400 is processedby the application software 410 of the STB. The current code pointvalues 430 for the metadata are then calculated and sent back to theserver side 480 to be compared with the desired code point values 470 tovalidate that the user interface data is correct or to take correctiveaction if necessary. The STB can then render 420 the metadata 400 usingthe appropriate fonts and graphics to provide a polished user interfaceto the user of the system.

In another alternative embodiment, the code point monitor may beimplemented within the subscriber's STB itself. In this embodiment, whenthe data is received the STB may perform code checks on it before orwhile the STB is displaying the GUI data or playing the video or audio.The STB may also perform all the post-error detection actions that werepreviously explained, including data correction, content blocking, anderror logging and reporting.

The code point monitor may be configured to perform the code checks onevery piece of content or to perform the checks only under certainconditions. For example, the code point monitor may check a data packageonly the first time it is loaded into the system or the first time thatit is accessed. In another example, the code point monitor may performrandom checks on data packages. In another example, the code pointmonitor may perform code checks only when the hardware on which it isoperating has excess processor time (e.g., the processor has idleprocessor time that it is not using for other tasks). The code pointmonitor may also only perform code checks when an operator or subscriberrequests that a check be performed, for example.

Certain code checks may not require a code point to be included. Forexample, if the text in the package contains only typical English wordsthen a spell check code check could be performed using any standardspell check dictionary. The absence of any desired code point for aspell check could then indicate that if any word in the text is notpresent in the dictionary then an error has occurred in the data.

The present invention is not limited to the specific embodimentsdescribed. It is expected that those skilled in the art will be able todevise other implementations that embody the principles of the presentinvention and remain within its scope. For example, although explainedhere in regard to interactive television applications, the presentinvention may also be applied in other situations where data packagesare transmitted over a network and the integrity of the data packagecannot be ensured, such as applications which send data packages overthe Internet. Furthermore, although explained here in regard to a cabletelevision network, the present invention may also be used with anyother kind of networks where it is important to be able to detect errorsand respond to them. For example, the present invention may also be usedwith wireless networks such as satellite transmission or cellulartransmission. The network used with the present invention may alsoemploy combinations of networking technologies.

1.-19. (canceled)
 20. A system for detecting changes in a data packagetransmitted over a network, comprising: a data network; a data packageserver connected to the data network; and a code point monitor connectedto the network, wherein the data package server is configured todetermine a set of desired code point values based on the data packagebefore transmitting the data package and the set of desired code pointvalues over the data network, and the code point monitor is configuredto compare the set of desired code point values to a set of current codepoint values determined after the data package has been received. 21.The system of claim 20, further comprising: a subscriber set top box,wherein the subscriber set top box is configured to receive the datapackage from the data package server and to determine the set of currentcode point values based on the received data package.
 22. The system ofclaim 21, wherein the subscriber set top box includes the code pointmonitor.
 23. The system of claim 21, wherein the subscriber set top boxis configured to transmit the set of current code point values to aseparate code point monitor.
 24. The system of claim 20, wherein thecode point monitor is configured to log the result of the comparison.25. The system of claim 20, wherein the code point monitor is configuredto determine if a change has occurred in the data package based on thecomparison of the sets of code point values; and the code point monitoris configured to transmit an instruction to perform a corrective actionto a network component if a change is determined to have occurred. 26.The system of claim 25, wherein the corrective action is preventing useof at least a portion of the data package.
 27. The system of claim 25,wherein the corrective action is requesting retransmission of at least aportion of the data package.
 28. The system of claim 25, wherein thecode point monitor determines a particular data element in which thechange occurred, and the corrective action is based on characteristicsof the particular data element in which the change occurred.
 29. Thesystem of claim 20, wherein the set of desired code point values aretransmitted with the data package.
 30. The system of claim 20, whereinthe set of desired code point values are transmitted separately from thedata package.
 31. A code point monitor comprising: a network interface;and a processor configured to: receive at least two data elements;receive, separately from the at least two data elements, a set ofdesired code point values corresponding to a data package; determine aset of current code point values for the at least two data elements; andcompare the set of current code point values to the set of desired codepoint values.
 32. The code point monitor of claim 31, wherein theprocessor is further configured to: determine if a change has occurredin the data package based on the comparison of the sets of code pointvalues; and if a change is determined to have occurred, requestretransmission of at least one of the at least two data elements. 33.The code point monitor of claim 31, wherein the processor is furtherconfigured to: determine if a change has occurred in the data packagebased on the comparison of the sets of code point values; and if achange is determined to have occurred, prevent the use of at least oneof the at least two data elements.
 34. The code point monitor of claim31, wherein the processor is further configured to: determine if achange has occurred in the data package based on the comparison of thesets of code point values; and if a change is determined to haveoccurred: identify the particular data element, of the at least two dataelements, which caused the error, and determine a corrective action toperform.
 35. The code point monitor of claim 34, wherein the correctiveaction is requesting a retransmission of at least one of the at leasttwo data elements.
 36. The code point monitor of claim 34, wherein thecorrective action is preventing the use of at least one of the at leasttwo data elements.
 37. The code point monitor of claim 34, wherein thecorrective action to perform is determined based on characteristics ofthe particular data element in which the change occurred.
 38. The codepoint monitor of claim 31, wherein the processor is further configuredto: determine if a change has occurred in the data package based on thecomparison of the sets of code point values; and if a change isdetermined to have occurred: identify the particular data element, ofthe at least two data elements, in which the change occurred, and basedon the similarity of a watermark in the particular data element in whichthe change occurred to a corresponding watermark in the set of desiredcode point values, determine that the particular data element may stillbe used.
 39. The code point monitor of claim 37, wherein the correctiveaction is determined based on the particular data element in which thechange occurred being related to a content rating of the data package.